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EXECUTIVE  SUMMARY 


Background 

Attempts  to  identify  buried  objects  using  conventional  geophysical  survey  data  have  produced 
uniformly  unsatisfactory  results.  The  problem  is  that  classification  based  on  dipole  inversion  of 
mapped  survey  data  is  intolerant  of  even  centimeter-scale  positioning  errors  in  the  data,  and  the 
technology  for  geo-location  of  survey  data  cannot  provide  the  required  positioning  accuracy.  As 
originally  envisioned,  this  project  sought  to  demonstrate  improved  procedures  for  target 
classification  with  EM61  survey  data  using  schemes  that  are  tolerant  of  the  positioning  errors. 
Several  procedures  were  tested  with  EM61  data  from  the  ESTCP  Classification  Demonstration  at 
the  former  Camp  Beale  in  California.  They  showed  no  improvement  in  classification 
performance  over  the  results  using  standard  processing  techniques.  Consequently,  the  project 
was  re-directed  to  consider  classification  perfonnance  using  survey-mode  data  collected  using  an 
advanced  man-portable  (MP)  electromagnetic  induction  (EMI)  sensor  array  recently  developed 
by  the  Chemistry  Division  of  the  Naval  Research  Laboratory  (NRL)  and  SAIC. 

The  MP  system  uses  a  cart  mounted  2x2  array  of  EMI  sensors  with  tri-axial  receiver  cubes.  The 
success  of  the  MP  system  for  cued  target  identification  in  the  Camp  Beale  demonstration  was  the 
primary  motivating  factor  for  adapting  the  system  for  dynamic  or  survey  mode  operation  in  this 
project. 

Objective 

The  objective  of  this  demonstration  was  to  validate  the  perfonnance  of  the  MP  system  used  in 
dynamic  survey  mode  in  a  blind  test  at  a  live  munitions  response  (MR)  site.  Performance  metrics 
include  production  rate,  detection  performance,  percentage  of  targets  classified  using  survey  data 
and  classification  performance. 

Results 

The  demonstration  was  part  of  the  ESTCP  Live  Site  Demonstration  at  the  fonner  Spencer 
Artillery  Range,  TN  during  May  2012.  The  dynamic  test  area  covered  0.5  Ha  of  open  field.  We 
report  the  classification  performance  results  for  339  unknown  anomalies  detected  within  the 
Dynamic  Area.  Approximately  70  percent  of  the  detected  anomalies  could  be  classified  from  the 
dynamic  data  alone.  Using  a  combination  of  results  from  dynamic  data  and  cued  data,  100%  of 
the  identified  UXO  were  correctly  classified  and  the  number  of  necessary  digs  could  be  reduced 
by  at  least  75  percent. 

Implementation  Issues 

The  objective  of  this  project  was  to  demonstrate  a  UXO  classification  process  that  made  use  of 
dynamic  mode  data  collection  with  an  advanced  EMI  sensor.  The  data  collected  with  these 
systems  can  be  used  both  for  anomaly  detection  and  for  classification  on  a  significant  fraction  of 
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the  detected  anomalies,  limiting  the  number  of  anomalies  requiring  further  investigation  in  cued 
mode. 

Another  ongoing  goal  of  this  and  other  projects  has  been  to  transition  these  technologies  from 
being  research  prototypes  to  use  in  the  industrial  community  where  appropriate.  The  mechanics 
of  collecting  classification-grade  advanced  EMI  cued  data  with  these  systems  have  been  shown 
to  be  fairly  routine  in  the  research  community.  As  part  of  the  ESTCP  Munitions  Response  Live 
Site  Demonstrations,  industrial  partners  have  been  exposed  to  the  MP  system  and  the  associated 
data  collection  and  processing  procedures.  The  success  of  this  effort  will  be  evaluated  on  an 
ongoing  basis  through  the  Live  Site  demonstrations.  In  the  past,  analysis  of  data  from  these 
systems  has  been  somewhat  of  a  specialty,  requiring  specific  software  and  knowledge  to 
proficiently  conduct.  The  successful  transition  of  the  processing  and  analysis  procedures  for  MP 
data  to  the  Geosoft  Oasis  montaj  environment  provides  a  clear  pathway  forward. 
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1.0  INTRODUCTION 


1.1  BACKGROUND 

The  characterization  and  remediation  activities  conducted  at  Department  of  Defense  sites 
contaminated  with  unexploded  ordnance  (UXO)  using  traditional  geophysical  sensors  such  as  the 
Geonics  EM61  often  yield  unsatisfactory  results  and  are  too  expensive.  In  part,  this  is  due  to  the 
inability  of  that  sensor  technology  to  distinguish  between  UXO  and  non-hazardous  clutter.  Field 
experience  cited  by  the  Corps  of  Engineers  is  that  seldom  more  than  1%  or  2%  of  the  items 
excavated  at  a  site  are  UXO  [1]. 

Attempts  to  identify  buried  objects  using  conventional  geophysical  survey  data  have  produced 
uniformly  unsatisfactory  results.  The  problem  is  that  classification  based  on  dipole  inversion  of 
mapped  survey  data  is  intolerant  of  even  centimeter-scale  positioning  errors  in  the  data,  and  the 
technology  for  geo-location  of  survey  data  cannot  provide  the  required  positioning  accuracy 
[2,  3,  4,  5,  6].  As  originally  envisioned,  this  project  sought  to  demonstrate  improved  procedures 
for  target  classification  with  EM61  survey  data  using  schemes  that  are  tolerant  of  the  positioning 
errors.  Several  procedures  were  tested  with  EM61  data  from  the  Environmental  Security 
Technology  Certification  Program  (ESTCP)  Classification  Demonstration  at  the  former  Camp 
Beale  in  California  [7].  They  showed  no  improvement  in  classification  performance  over  the 
results  using  standard  processing  techniques  which  were  reported  in  reference  7.  Consequently, 
the  project  was  re-directed  to  consider  classification  performance  using  survey-mode  data 
collected  using  an  advanced  man-portable  (MP)  electromagnetic  induction  (EMI)  sensor  array 
recently  developed  by  the  Chemistry  Division  of  the  Naval  Research  Laboratory  (NRL)  and 
SAIC. 

NRL  and  SAIC  have  participated  in  several  programs  funded  by  the  Strategic  Environmental 
Research  and  Development  Program  (SERDP)  and  ESTCP  whose  goal  has  been  to  enhance  the 
classification  ability  of  the  Multi-sensor  Towed  Array  Detection  System  (MTADS).  The  NRL 
Time-domain  Electromagnetic  MTADS  (TEMTADS)  vehicle  towed  5x5  array  incorporated  an 
advanced  EMI  sensor  specifically  designed  for  UXO  classification  [8]. 

This  technology  was  transitioned  to  smaller  systems  for  deployment  in  more  confined  areas  in 
ESTCP  Projects  MR-200807  and  200909  [9].  The  man-portable  (MP)  system  was  constructed 
as  a  2x2  array  of  upgraded  sensors  based  on  those  from  the  original  TEMTADS,  but  with  tri- 
axial  receiver  cubes.  The  success  of  the  MP  system  for  cued  target  identification  in  the  Camp 
Beale  demonstration  [7]  was  the  primary  motivating  factor  for  adapting  the  system  for  dynamic 
or  survey  mode  operation  in  this  project. 

1.2  OBJECTIVES  OF  THE  PROJECT 

The  objective  of  this  demonstration  was  to  validate  the  perfonnance  of  the  MP  system  used  in 
dynamic  mode  through  blind  testing  at  a  live  site.  The  dynamic  MP  system  results  from  the 
ESTCP  Munitions  Response  (MR)  Live  Site  Demonstration  at  the  former  Spencer  Artillery 
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Range,  located  in  Spencer,  TN  in  May  2012  are  presented  in  this  document.  To  limit  the 
repetition  of  infonnation,  study-  and  site-specific  information  that  are  presented  elsewhere,  such 
as  in  the  ESTCP  Live  Site  Demonstrations  Plan  [10],  are  noted  and  not  repeated  in  this 
document. 

The  MP  system  was  evaluated  in  terms  of  classification  performance  (false  alarm  rejection)  and 
appropriateness  for  fielding  (production  rate,  etc.).  Specifics  are  provided  in  Section  3.0. 

1.3  REGULATORY  DRIVERS 

Stakeholder  acceptance  of  the  use  of  classification  techniques  on  real  sites  will  require 
demonstration  that  these  techniques  can  be  deployed  efficiently  and  with  high  probability  of 
discrimination.  Demonstration  at  live  sites  with  extensive  ground-truth  validation  will  facilitate 
regulatory  acceptance  of  the  UXO  classification  technology  and  methodology. 

2.0  TECHNOLOGY 

2.1  TECHNOLOGY  DESCRIPTION 

The  MP  system  comprises  a  square  array  of  four  transmit(Tx)/receive(Rx)  coil  pairs  mounted  on 
a  cart.  The  transmit  coils  are  wound  around  the  outer  portion  of  35  cm  square  Styrofoam  forms. 
The  three-axis  receiver  cubes  are  wound  on  8  cm  wooden  blocks.  Figure  2-1  (left)  shows  a  new 
coil  under  construction.  Figure  2-1  (right)  shows  the  MP  cart  with  GPS  antenna.  The  TEM  and 
data  acquisition  electronics  are  in  backpack  worn  by  the  MP  cart  operator.  Data  acquisition  is 
controlled  by  the  tablet  computer  carried  by  the  person  walking  along  to  the  operator’s  right. 


Figure  2-1  -  Left:  Individual  TEMTADS/3D  EMI  sensor  with  3-axis  receiver  under 
construction,  Right:  Man-portable  TEM  array  array  with  GPS  antenna. 

For  dynamic  survey  mode  operation  we  used  a  decay  time  (and  corresponding  transmitter  on 
time)  of  2.77  ms.  A  base  time  period  of  33  ms  was  used,  so  that  three  repeats  per  transmit 
wavefonn  are  averaged.  Gate  width  was  set  at  20%,  resulting  in  19  time  gates  with  center  times 
ranging  from  25  ps  to  2.5  ms. 
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2.2  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 


The  MP  system  was  originally  designed  to  offer  similar  cued-mode  production  rates  to  those 
seen  for  larger,  vehicular-towed  advanced  EMI  sensors  while  able  to  operate  in  difficult  terrain 
and  treed  areas  that  the  larger  systems  cannot  access.  With  the  upgraded  TEMTADS/3D 
sensors,  similar  performance  was  achieved  with  similar  classification-grade  data  quality.  The 
MP  array  is  80  cm  on  a  side  and  mounted  on  a  man-portable  cart.  Terrain  where  the  vegetation 
or  topography  interferes  with  passage  of  a  cart  of  that  size  will  not  be  amenable  to  the  use  of  the 
system. 

There  is  a  limiting  anomaly  density  above  which  the  response  of  individual  targets  cannot  be 
separated  individually.  We  have  chosen  relatively  small  sensors  for  this  array  which  help 
mitigate  this  problem  but  we  cannot  eliminate  it  completely.  Recent  developments,  including 
solvers  designed  for  classification  in  multiple-object  scenarios  such  as  S AIC’s  multi-target 
solver,  [11]  are  being  evaluated  and  their  performance  characteristics  in  cluttered  environments 
detennined. 

In  dynamic  mode,  the  MP  system  offers  higher  data  density  and  correspondingly  finer  resolution 
of  targets  than  is  typically  seen  for  systems  with  larger  transmitter  and  receiver  coils,  such  as  the 
iconic  Geonics  EM61-MK2,  although  depths  of  detection  and  signal-to-noise  ratios  (SNR)  are 
comparable  between  the  EM61-MK2  and  the  MP  system.  However,  this  rich  data  set  comes  at  a 
productivity  cost.  A  complete  transmit  cycle  of  the  MP  system  in  dynamic  mode  has  a  repetition 
rate  of  7.5  Hz.  Systems  with  few  transmitters  can  cycle  faster,  resulting  in  higher  along-track 
data  density.  Recent  advances  in  smart,  or  dipole-based,  target  picking  indicate  that  the 
additional  richness  of  data  collected  with  the  advanced  sensors,  if  used  to  its  full  potential,  could 
improve  detection  performance  beyond  that  of  traditional  technologies. 

3.0  PERFORMANCE  OBJECTIVES 

Perfonnance  objectives  for  the  demonstration  are  summarized  in  Table  3-1.  They  provide  a  basis 
for  evaluating  the  performance  and  costs  of  the  demonstrated  technology. 

Production  rate  was  detennined  by  the  area  covered  during  a  10  hour  work  day.  This  includes 
required  setup,  calibration,  data  download  and  shut  down  activities  as  well  as  down  time  for 
lunch,  etc.  The  goal  was  a  production  rate  of  at  least  1  acre/day.  May  8  was  the  only  full  day  of 
data  collection.  Data  collection  (including  IVS)  ran  from  9:25AM  to  6:08PM.  During  that  time 
156  survey  data  lines  were  collected,  which  is  77%  of  the  202  total  survey  lines  used  to  cover  the 
1.3  acre  dynamic  survey  area.  Figuring  in  H4  hours  for  setup  and  shut  down  activities  this 
conesponds  to  an  area  coverage  rate  of  1  acre/day. 
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Table  3-1  -  Performance  Results  for  this  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Success? 

(Yes/No) 

Production  rate 

Area  per  unit  time 

Survey  data 

1  acre/day 

Yes 

Detection  of  all 
targets  of  interest 
(TOI) 

Percent  of  seeded 
items  detected 

Location  of 
seeded  items 
Anomaly  list 

100%  of  seeded 
items  detected 
within  a  60  cm  halo 

No 

Classification 
perfonnance 
(survey  mode) 

Percent  detected 
targets  classified 
with  survey  data 

Dynamic 
inversion  fit 
quality 

50%  anomalies 
classified  using 
survey  data 

Yes 

Classification 

Perfonnance 

(overall) 

Percent  clutter 
rejected  at  100% 

TOI  correctly 
identified 

Dig  list  and 
ground  truth 

75%  clutter 
rejection 

Yes 

The  system  must  be  capable  of  detecting  the  targets  of  interest  (TOI).  Seed  items  (inert 
munitions  and  pipe  sections)  buried  at  the  site  were  used  to  evaluate  detection  perfonnance.  The 
objective  was  considered  to  be  met  if  100%  of  the  seeded  items  were  detected  within  a  halo  of  60 
cm.  At  the  completion  of  the  dynamic  survey  of  the  Dynamic  Area,  a  target  list  was  produced 
using  the  criteria  outlined  in  Section  6.2.  As  this  was  the  first  live-site  demonstration  of  this 
sensor  in  this  mode  of  operation,  a  data  analyst  manually  evaluated  each  target  selection.  The 
resulting  target  list  was  submitted  to  the  Program  Office  for  evaluation  by  the  USACoE, 
Huntsville.  One  seed  item  was  missed  by  the  data  analyst  even  though  the  data  for  that  location 
met  the  selection  criteria.  A  root-cause-analysis  detennined  the  threshold  exceedance  for  the  late 
time  gate  was  not  well-formed  and  discarded  by  the  data  analyst.  With  the  aggressive  schedule 
required  for  this  demonstration,  fatigue  and  time  pressure  on  the  data  analyst  played  an 
additional  role.  In  future  demonstrations,  an  automated  version  of  the  target  picking  process  will 
be  used  and  will  prevent  this  type  of  error. 

Our  goal  is  to  significantly  reduce  the  number  of  anomalies  which  must  be  revisited  for 
classification.  Substantial  cost  savings  can  be  realized  when  a  significant  percentage  of  the 
anomalies  can  be  classified  using  the  survey  data.  The  objective  was  considered  met  if  50%  of 
the  anomalies  could  be  classified  using  the  survey  data.  232  (68%)  of  the  339  anomalies  were 
classified  using  only  the  survey  data.  Cued  data  were  requested  for  the  other  107  anomalies. 

The  goal  of  classification  is  to  significantly  reduce  the  number  of  unnecessary  clutter  digs 
without  leaving  any  TOI  in  the  ground.  The  metric  for  the  final  objective  was  the  percentage  of 
clutter  anomalies  that  were  correctly  classified  with  all  TOI  correctly  classified.  The  data  analyst 
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set  a  “stop-dig”  threshold  at  94  total  digs  which  captured  all  23  of  the  TOI  (Pd  =  1.0)  in  the 
Dynamic  Area.  70  of  the  316  clutter  items  in  the  Dynamic  Area  (22%)  would  have  been  dug  at 
this  “stop-dig”  threshold,  corresponding  to  a  78%  reduction  in  clutter  digs. 

4.0  SITE  DESCRIPTION 

The  infonnation  in  this  section  is  extracted  from  the  ESTCP  Live  Site  Demonstrations  Plan  [10]. 

4.1  SITE  SELECTION 

This  site  was  chosen  by  ESTCP  in  their  series  of  sites  for  demonstration  of  the  classification 
process.  The  first  site  in  the  series,  former  Camp  Sibert  in  Alabama,  had  only  one  TOI  and  item 
“size”  was  an  effective  discriminant.  A  hillside  range  at  the  former  Camp  San  Luis  Obispo  in 
California  was  selected  for  the  second  of  these  demonstrations  because  of  the  wider  mix  of 
munitions,  including  60  mm,  81  mm,  and  4.2-in  mortars  and  2.36-in  rockets.  Three  additional 
munitions  types  were  discovered  during  the  course  of  the  demonstration.  The  third  site  chosen 
was  the  former  Camp  Butner  in  North  Carolina.  This  site  is  contaminated  with  items  as  small  as 
37  mm  projectiles,  adding  yet  another  layer  of  complexity  into  the  process.  Additional  sites 
including  this  one  provide  opportunities  to  demonstrate  the  capabilities  and  limitations  of  the 
classification  process  on  a  variety  of  site  conditions. 

This  site  was  selected  for  demonstration  because  it  is  more  heavily  wooded  than  prior 
demonstrations  and  is  thought  to  contain  a  wide  mixture  of  munitions.  These  two  features 
increase  the  site’s  complexity  and  both  characteristics  are  likely  to  be  encountered  on  production 
sites.  A  1.3  acre  open  area  of  the  site  was  chosen  for  the  dynamic  survey  demonstrations. 

4.2  SITE  HISTORY 

In  1941,  construction  began  on  the  30,618  acre  Spencer  Artillery  Range  and  documentation 
identifies  establishment  of  two  impact  areas:  Jakes  Mountain  (5,060  acres)  and  Bald  Knob 
(2,090  acres).  Troop  training  took  place  until  September  1944,  by  which  time  Army  ground 
forces  had  either  departed  or  were  under  orders  to  depart.  Subsequent  arrangements  were  made 
for  Dyersburg  Army  Air  Field  to  use  the  Spencer  Artillery  Range  as  an  air-to-ground  gunnery 
range.  The  land  reverted  back  to  the  original  25  leaseholders  in  the  summer  of  1946.  Several 
surface  decontamination  sweeps  were  completed  on  portions  of  the  fonner  range  in  the  1950s. 
Since  then,  numerous  tracts  of  land  have  been  sold  and/or  subdivided,  significantly  increasing 
the  number  of  property  owners  from  the  original  25  to  several  hundred  landowners  today. 

4.3  MUNITIONS  CONTAMINATION 

The  suspected  munitions  at  this  site  include  37  mm  projectiles,  75  mm  projectiles,  76  mm 
projectiles,  105  mm  projectiles  and  155  mm  projectiles.  In  close  proximity  to  the  particular  site 
of  this  demonstration,  37  mm  and  155  mm  projectiles  were  observed  during  the  Remedial 
Investigation  as  well  as  large  quantities  of  unidentified  munitions  debris. 
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5.0  TEST  DESIGN 


5.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

The  dynamic  MP  system  survey  that  is  the  focus  of  this  report  was  part  of  a  larger  TEMTADS 
family  demonstration,  as  discussed  in  Reference  12.  The  basic  idea  here  was  to: 

1 .  Conduct  a  geophysical  survey  of  the  Spencer  Dynamic  Area  using  the  MP  system, 

2.  Select  anomalies  from  the  mapped  survey  data  consistent  with  expected  signal  levels  for 
TOI, 

3.  Classify  as  many  anomalies  as  possible  using  the  survey  data,  and 

4.  Use  static  cued  data  collected  with  the  MP  system  to  classify  the  remaining  anomalies. 

A  dig  list  based  on  the  classification  data  would  then  be  submitted  to  the  ESTCP  Program  for 
scoring  to  detennine  classification  performance. 

5.2  SITE  PREPARATION 

The  Dynamic  Area  was  recently  harvested  of  trees.  To  prepare  the  area  for  the  survey,  tree 
stumps  were  ground  and  remaining  vegetation  removed.  All  visible  metal  objects  were  removed 
from  the  surface  at  the  final  selected  demonstration  site. 

At  a  live  site  such  as  this,  the  ratio  of  clutter  to  TOI  is  such  that  only  a  small  number  of  TOI  may 
be  found;  far  from  enough  to  determine  any  demonstrator’s  classification  perfonnance  with 
acceptable  confidence  bounds.  To  avoid  this  problem,  the  site  was  seeded  by  the  ESTCP 
Program  Office  with  enough  TOI  to  ensure  reasonable  statistics. 

5.3  SYSTEM  SPECIFICATIONS 

The  MP  system  has  four  of  the  TEMTADS/3D  EMI  sensors  described  in  Section  2.1  arranged  in 
a  2x2,  80  cm  square  array.  The  array  is  deployed  on  a  wheeled  cart  fabricated  from  PVC  plastic 
and  G-10  fiberglass  as  shown  in  Figure  2-1  (right).  The  sensor  ride  height  is  20  cm.  The  MP 
system  can  be  operated  in  two  modes:  dynamic  (or  survey)  mode  and  cued  mode.  In  dynamic 
mode,  a  GPS  antenna  and  (optionally)  an  inertial  measurement  unit  (IMU)  are  mounted  above 
the  TEM  array. 

The  transmitter  electronics  and  the  data  acquisition  computer  are  mounted  in  the  operator 
backpack.  The  EM3D  software  package,  a  variant  of  the  software  used  for  the  Geometries 
MetalMapper,  provides  dynamic  data  acquisition  functionality.  Each  transmitter  is  fired  in  a 
sequence.  The  received  signal  is  recorded  for  all  Rx  channels  for  each  transmit  cycle.  The 
transmit  pulse  waveform  duration  is  33  ms  in  dynamic  mode.  While  it  is  possible  to  record  the 
entire  decay  transient  at  2  ps  sample  spacing,  we  have  found  that  binning  the  data  into  time  gates 
simplifies  the  analysis  and  provides  additional  signal  averaging  without  significant  loss  of 
temporal  resolution  in  the  transient  decays  [13].  In  dynamic  mode,  the  data  are  binned  into  19 
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logarithmically  spaced  time  gates.  The  data  are  recorded  in  a  binary  format  as  a  single  file  with 
multiple  data  points  (one  data  point  per  Tx  cycle). 

The  data  acquisition  computer  is  mounted  in  the  backpack  and  is  controlled  by  a  second  operator 
using  a  tablet  PC  with  a  IEEE  802. 1 1  g  wireless  link  to  the  data  acquisition  computer.  The 
second  operator  also  manages  field  notes  and  team  orienteering  functions.  Data  collection  with 
the  MP  system  at  the  former  Spencer  Artillery  Range,  TN  is  shown  in  Figure  2-1  (right). 

Positioning  is  provided  using  cm-level  Real  Time  Kinematic  (RTK)  Global  Positioning  System 
(GPS)  receivers.  To  achieve  cm-level  precision,  a  fixed  reference  base  station  is  placed  on  an 
established  first-order  survey  control  point  near  the  survey  area.  The  base  station  transmits 
corrections  to  the  GPS  rover  at  1  Hz  via  a  radio  link  (450  MHz).  The  rover  GPS  receiver 
receives  corrections  from  the  fixed  base  station.  This  corrected  position  is  reported  at  10-20  Hz 
using  a  vendor-specific  NMEA-0183  message  fonnat  ( e.g .  $PTNL,GGK ).  The  RTK  receiver  is 
mounted  above  the  array  center  on  a  tripod. 

5.4  CALIBRATION  ACTIVITIES 

The  system  is  calibrated  by  comparing  the  measured  response  to  a  standard  4-inch  diameter 
aluminum  ball  with  the  expected  response  calculated  using  standard  EMI  theory  [14]. 

5.5  DATA  COLLECTION  PROCEDURES 

The  Dynamic  Area  was  a  1.3  acre  section  of  the  Spencer  Range  demonstration  site.  Geophysical 
surveys  of  the  Dynamic  Area  were  conducted  with  the  MP  system  and  a  Geonics  EM61.  The 
Program  Office  selected  a  total  of  339  anomalies  from  the  combined  surveys  as  potential  TOI. 
Of  these,  23  were  TOI  and  316  were  clutter.  Performance  of  the  system  response  was  monitored 
on  a  twice-daily  basis  using  the  onsite  instrument  verification  strip  (IVS). 

The  sensor  spacing  for  the  TEM  array  is  fixed  at  40  cm  in  both  along-  and  cross-track  directions 
by  design.  In  dynamic  mode  a  complete  Tx  cycle  (sequentially  firing  each  of  the  four 
transmitters)  occurs  every  0.13  s  (7.5  Hz).  At  a  walking  speed  of  ~1  m/s  this  corresponds  to  7.5 
complete  Tx  cycles  per  meter.  Survey  lines  are  spaced  every  40  cm. 

Data  were  stored  electronically  as  collected  on  the  data  acquisition  computer  hard  drive. 
Approximately  every  survey  hour,  the  collected  data  were  copied  onto  removable  media  and 
transferred  to  the  data  analyst  for  quality  control  (QC)  and  subsequent  analysis.  The  data  were 
moved  onto  the  data  analyst’s  computer  and  the  media  was  recycled.  Raw  data  and  analysis 
results  were  backed  up  from  the  data  analyst’s  computer  to  external  hard  disks  daily.  These 
results  were  archived  on  an  internal  file  server  at  NRL  or  SAIC  at  the  end  of  the  survey. 

At  the  conclusion  of  data  collection  activities,  all  anomalies  on  the  master  anomaly  list 
assembled  by  the  Program  Office  were  excavated.  Each  item  encountered  was  identified, 
photographed,  its  depth  measured,  its  location  detennined  using  cm-level  GPS,  and  the  item 
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removed  if  possible.  This  ground  truth  information  was  used  to  validate  the  objectives  listed  in 
Section  3.0. 

6.0  DATA  ANALYSIS  AND  PRODUCTS 

6.1  PREPROCESSING 

Prior  to  detection  and  classification  processing  the  data  were  nonnalized  by  transmit  current, 
edited  to  remove  noise  spikes  and  leveled  using  a  median  filter. 

6.2  TARGET  SELECTION  FOR  DETECTION 

An  anomaly  detection  procedure  similar  to  the  one  described  in  Reference  15  was  used  for  the 
MP  system  dynamic  survey  data.  As  this  was  the  first  outing  of  the  MP  system  in  dynamic 
mode,  a  data  analyst  made  each  anomaly  selection  rather  than  an  automated  peak  picker  routine. 
The  anomaly  detection  criteria  were  unchanged.  A  preliminary  detection  threshold  was  selected 
based  on  physical  models  of  the  systems  response  to  the  expected  TOI.  The  site-specific 
background  signal  levels  were  considered  as  well.  Anomalies  were  picked  from  mapped  data. 
The  mapped  data  from  the  Dynamic  Area  are  shown  in  Figure  6-1.  The  data  presented  are 
monostatic  response  from  each  sensor  at  the  tenth  usable  time  gate,  1.024  ms. 

The  ESTCP  Demonstration  Plan  for  the  fonner  Spencer  Artillery  Range  demonstration  [10]  set 
an  objective  of  detecting  37  mm  projectiles  to  a  burial  depth  of  34  cm.  To  establish  a  detection 
threshold  for  this  objective  with  the  MP  system  operating  in  dynamic  survey  mode,  a  series  of 
forward  model  cases  were  run  using  the  polarizabilities  of  known  37  mm  projectiles  and  actual, 
measured  survey  track  positions  from  our  test  field.  In  dynamic  survey  mode,  the  earliest  usable 
time  gates  are  in  the  0.1  to  0.2  ms  range.  Therefore,  the  first  time  gate  considered  in  the  forward 
model  cases  was  0. 135  ms.  The  weakest  responses  are  37  mm  projectiles  oriented  horizontally. 

A  forward  model  was  run  with  a  fixed  object  depth  of  34  cm,  but  over  a  range  of  object  -  survey 
tracks  separations  and  a  range  of  object  azimuth  orientations.  The  results  indicated  that  the 
expected  peak  signals  for  the  37  mm  projectile  are  found  within  the  range  of  1.6  to  2.1  mV/A  at 
0.135  ms.  Based  on  these  modeling  results,  a  pre-demonstration,  conservative  detection  level  of 
1.8  mV/A  was  selected  for  the  MP  system  dynamic  survey. 

After  the  data  were  collected  and  reviewed,  it  was  detennined  that  the  last  time  gate  (of  10  used 
for  analysis)  was  a  better  choice  for  target  picking.  The  response  from  small,  thin-walled  items 
has  had  a  chance  to  decay  away  while  the  response  from  a  37  mm  projectile  still  remained 
sufficiently  above  background  for  target  picking.  The  same  model  used  above  was  used  to 
detennine  the  proper  threshold  of  the  10th  time  gate  and  was  found  to  be  0. 18  mV/A.  Therefore, 
we  decided  to  use  the  10th  time  gate  for  initial  selection  and  the  1st  time  gate  with  the 
corresponding  model  prediction  in  the  range  of  1.6  to  2.1  mV/A  threshold  to  confirm  our  picks. 
This  threshold  is  within  the  range  stipulated  in  the  original  plan.  If  a  peak  passed  the  threshold  at 
both  time  gates,  it  was  added  to  the  target  list. 
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Figure  6-1  -  Located  and  leveled  dynamic  data  (1.024  ms)  from  the  MP  system  for  the 
Dynamic  Area  at  the  former  Spencer  Artillery  Range,  TN. 

6.3  PARAMETER  ESTIMATION 

The  raw  signature  data  from  TEMTADS  sensors  reflect  details  of  the  sensor/target  geometry  as 
well  as  inherent  EMI  response  characteristics  of  the  targets  themselves.  In  order  to  separate  out 
the  intrinsic  target  response  properties  from  sensor/target  geometry  effects,  we  invert  the 
signature  data  to  estimate  principal  axis  magnetic  polarizabilities  for  the  targets.  The 
TEMTADS  data  are  inverted  using  the  standard  induced  dipole  response  model  wherein  the 
effect  of  eddy  currents  set  up  in  the  target  by  the  primary  field  is  represented  by  a  set  of  three 
orthogonal  magnetic  dipoles  at  the  target  location  [16]. 


9 


Given  a  set  of  measurements  of  the  target  response  with  varying  geometries  or  "look  angles"  at 
the  target,  the  data  can  be  inverted  to  detennine  the  local  (X,Y,Z)  location  of  the  target,  the 
orientation  of  its  principal  axes  (4>,0,\(/),  and  the  principal  axis  polarizabilities  (Pi,P2,P3).  The 
basic  idea  is  to  search  out  the  set  of  nine  parameters  (X,Y,Z,c|),0,\|/,Pi,P2,P3)  that  minimizes  the 
difference  between  the  measured  responses  and  those  calculated  using  the  dipole  response 
model. 

6.4  CLASSIFIER  AND  TRAINING 

Target  classification  is  based  on  a  library  matching  procedure  wherein  we  compare  the  results  of 
a  dipole  inversion  of  the  TEM  array  data  to  principal  axis  polarizabilities  drawn  from  a  library  of 
known  signatures.  The  match  is  based  on  three  criteria:  the  amplitude  of  the  primary 
polarizability,  and  the  ratio  of  the  second  and  third  polarizabilities  to  the  first.  We  have 
computed  match  metrics,  each  of  which  runs  from  0  (terrible  match)  to  1  (perfect  match). 

Our  experience  with  these  sensors  has  been  that  principal  polarizabilities  determined  from  in-air 
measurements  are  indistinguishable  from  those  determined  from  measurements  taken  over  buried 
targets.  We  have  an  extensive  collection  of  inert  military  munitions  collected  from  many  sources 
which  were  measured  at  our  home  facility  using  the  TEMTADS  family  of  sensors  mounted  on  a 
test  stand.  We  have  also  assembled  a  fairly  extensive  polarizability  database  for  clutter  items 
recovered  from  several  different  sites.  These  data  collections  were  used  as  training  data  for 
establishing  UXO/clutter  discrimination  boundaries  on  the  library  match  metrics. 

6.5  DATA  PRODUCT  SPECIFICATIONS 

Provided  to  the  ESTCP  Program  Office  as  deliverables  were;  the  Dynamic  Area  survey  data,  test 
pit  data  for  the  site-specific  TOIs,  and  data  from  the  daily  IVS  surveys.  All  data  are  provided 
leveled  and  current-corrected  in  ASCII  file  fonnats.  For  the  test  pit  and  IVS  data,  the  raw  data 
files  are  also  provided.  See  Reference  12  for  further  detailed  data  product  specifications. 

7.0  PERFORMANCE  ASSESSMENT 

7.1  PRODUCTION  RATE 

Time  stamped  survey  data  files  were  used  to  detennine  the  survey  production  rate.  The  data 
collection  rate  is  the  product  of  lane  spacing  and  average  along-track  speed.  The  lane  spacing  for 
the  MP  survey  was  0.4  m  and  the  average  survey  speed  was  0.95  m/s,  resulting  in  a  data 
collection  rate  of  0.38  m2/s.  This  was  -80%  of  the  EM61  data  collection  rate  in  the  Spencer 
Dynamic  Area,  which  had  0.5  m  line  spacing  and  an  average  survey  speed  of  1.1  m/s.  It  amounts 
to  0.35  acre/hour. 

The  actual  area  coverage  or  production  rate  for  a  10  hour  day  factors  in  required  setup, 
calibration,  data  download  and  shut  down  activities,  as  well  as  down  time  for  lunch,  etc.  May  8 
was  the  only  full  day  of  data  collection.  Data  collection  (including  IVS)  ran  from  9:25AM  to 
6:08PM.  During  that  time  156  survey  data  lines  were  collected,  which  is  77%  of  the  202  total 
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survey  lines  used  to  cover  the  1.3  acre  dynamic  survey  area.  Figuring  in  1  %  hours  for  setup  and 
shut  down  activities  this  corresponds  to  an  area  coverage  rate  of  1  acre/day,  which  amounts  to 
only  about  30%  of  the  data  collection  rate. 

At  a  line  spacing  of  40  cm,  one  pair  of  receiver  cubes  for  each  survey  line  passes  roughly  over 
the  same  path  as  a  pair  of  receiver  cubes  from  the  survey  line  on  either  side.  In  more  recent 
demonstrations,  this  spacing  has  been  increased  to  60  cm,  improving  the  production  rate.  For 
very  small  targets,  such  as  20mm  projectiles,  the  original  spacing  is  still  recommended. 

7.2  ANOMALY  DETECTION 

At  the  completion  of  the  dynamic  survey  of  the  Dynamic  Area,  a  target  list  was  produced  using 
the  criteria  outlined  in  Section  6.2.  As  this  was  the  first  live-site  demonstration  of  this  sensor  in 
this  mode  of  operation,  a  data  analyst  manually  evaluated  each  target  selection.  The  resulting 
target  list  was  submitted  to  the  Program  Office  for  evaluation  by  the  Corps  of  Engineers, 
Huntsville.  One  seed  item  was  missed  by  the  data  analyst  even  though  the  data  coverage  at  that 
location  was  good  and  the  sensor  readings  met  the  selection  criteria.  A  root-cause-analysis 
detennined  the  threshold  exceedance  for  the  late  time  gate  was  not  well-formed  and  discarded  by 
the  data  analyst.  See  Figure  7-1.  Additionally,  the  centroids  of  the  peaks  at  the  early  and  late 
time  gates  did  not  line  up  well.  With  the  aggressive  schedule  required  for  this  demonstration, 
fatigue  and  time  pressure  on  the  data  analyst  played  a  role  in  this  failure.  In  future 
demonstrations,  an  automated  version  of  the  target  picking  process  will  be  used  and  will  prevent 
this  type  of  error. 

7.3  SURVEY  MODE  CLASSIFICATION 

Based  on  the  techniques  described  in  Section  6.0,  the  dynamic  data  set  collected  at  the  former 
Spencer  Artillery  Range  Dynamic  Area  in  May  2012  was  used  to  generate  an  anomaly  list.  The 
union  of  this  list  and  the  anomaly  list  from  the  EM61-MK2  survey  conducted  by  URS 
Corporation  personnel  in  April  2012  was  used  to  generate  an  overall  anomaly  list  for  the 
Dynamic  Area.  Dipole  inversion  to  detennine  target  parameters  for  classification  was  attempted 
using  the  survey  data  for  all  339  anomalies  on  this  target  list.  Classifiable  target  parameters  were 
able  to  be  extracted  for  232  (68%)  of  the  anomalies.  Cued  data  were  requested  for  the  other  107 
anomalies. 
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Figure  7-1  -  Data  contour  plots  for  the  early  selection  time  gate  (0.137  ms,  left)  and  the  late  selection 
time  gate  (1.024  ms,  right).  Position  is  given  in  local  coordinates  (meters). 

7.4  CLUTTER  REJECTION 

Once  data  analysis  was  complete,  a  ranked  dig  list  was  prepared  and  submitted  to  ESTCP  for 
scoring.  The  results  of  the  classification  process  are  presented  in  Figure  7-2  in  the  form  of  a 
receiver  operating  characteristic  (ROC)  curve.  The  ground  truth  for  18  anomalies  was  requested 
for  training.  These  are  shown  by  the  black  section  of  the  ROC.  The  data  analyst  set  a  “stop-dig” 
threshold  at  94  total  digs  (end  of  red  portion  of  ROC)  which  captured  all  23  of  the  TOI  (Pd  =  1.0) 
in  the  Dynamic  Area.  The  remaining  anomalies  (green  portion)  were  classified  as  likely  clutter. 
70  of  the  316  clutter  items  in  the  Dynamic  Area  (22%)  would  have  been  dug  at  the  “stop-dig” 
threshold,  corresponding  to  a  78%  reduction  in  clutter  digs. 


Figure  7-2  -  TEMTADS  MP  2x2  Cart  Dynamic  /  Cued  Classification  Results  for 
the  former  Spencer  Artillery  Range,  TN.  Classification  performed  by  SA1C. 
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8.0  COST  ASSESSMENT 


8.1  COST  MODEL 

The  cost  elements  tracked  for  this  demonstration  are  detailed  in  Table  8-1.  The  cost  elements  are 
based  on  a  model  recently  developed  for  cost  estimation  for  the  MP  system  at  Camp  Beale  in 
2011  [17].  The  model  assumes  a  two-person  field  crew  and  one  data  analyst.  While  the  MP 
system  is  not  currently  commercially  available,  an  estimated  daily  rental  rate  is  provided  for 
comparison  to  other  technologies.  The  rental  rate  is  based,  in  part,  on  the  costs  of  items 
purchased  in  prototype  quantities  (single  units)  and  would  presumably  decrease  significantly  if 
the  items  were  procured  at  production  quantity  levels.  The  data  analysis  level  of  effort  included 
in  the  dynamic  mode  model  is  based  on  projections  of  the  production  rate  that  will  be  achievable 
with  UX-Analyze  and  not  the  actual  production  rate  achieved  in  this  first  demonstration. 

8.2  COST  DRIVERS 

Two  factors  are  expected  to  be  strong  drivers  of  cost  for  this  technology  as  demonstrated.  The 
first  is  the  daily  production  rate  (number  of  anomalies  for  cued  mode,  number  of  acres  for 
dynamic  mode).  Higher  productivity  in  data  collection  equates  to  more  anomalies  investigated 
for  a  given  period  of  time  in  the  field.  The  time  required  for  conducting  data  quality  control  and 
analysis  can  be  significantly  higher  than  for  other,  more  traditional  methods  and  could  become  a 
cost  driver  due  to  the  time  involvement.  The  data  analysts  must  be  trained  to  handle  the  more 
complex,  and  richer,  data  sets  properly.  The  thoughtful  use  of  available  automation  techniques 
with  operator  QC  support  can  moderate  this  effect. 

8.3  COST  BENEFIT 

The  main  benefit  to  using  a  UXO  classification  process  is  cost-related.  The  ability  to  reduce  the 
number  of  non-hazardous  items  that  have  to  be  dug  or  have  to  be  dug  as  presumptively- 
hazardous  items  directly  reduces  the  cost  of  a  remediation  effort.  The  additional  infonnation  for 
anomaly  classification  provided  by  these  sensor  systems  provides  additional  information  for  the 
purposes  of  anomaly  classification.  If  there  is  buy-in  from  the  stakeholders  to  use  these 
techniques,  this  information  can  be  used  to  reduce  costs.  Successful  implementation  of  dynamic 
mode  surveying  has  the  potential  for  further  cost  reduction  by  limiting  the  number  of  trips  to  a 
given  area  required. 
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Table  8-1  -  TEMTADS  MP  2x2  Cart  Dynamic  Mode  Tracked  Costs 


Cost  Element 

Data  Tracked 

Cost 

Data  Collection  Costs 

Pre/Post  Survey 

Activities 

Component  costs  and  integration  costs 
•  Spares  and  repairs 

$3,500 

Cost  to  pack  the  array  and  equipment, 
mobilize  to  the  site,  and  return 

•  Personnel  required  to  pack 

•  Packing  hours 

•  Personnel  to  mobilize 

•  Mobilization  hours 

•  Transportation  costs 

$12,450 

1 

16 

3 

8 

$7,250 

Cost  to  assemble  the  system,  perform 
initial  calibration  tests 

•  Personnel  required 

•  Hours  required 

$780 

3 

2 

Survey  Costs 

Unit  cost  per  acre  investigated.  This 
will  be  calculated  as  daily  survey  costs 
divided  by  the  number  of  acres 
investigated  per  day. 

•  Equipment  Rental  (day) 

•  Daily  calibration  (hours) 

•  Survey  personnel  required 

•  Survey  hours  per  day 

•  Daily  equipment  break-down  and 
storage  (hours) 

$3,375  /  acre 

$190 

0.5 

2 

8 

0.5 

Processing  Costs 

$2,340  /  acre 

Preprocessing 

Time  required  to  perfonn  standard  data 
clean  up  and  to  merge  the  location  and 
geophysical  data. 

7.5  hr/acre 

Target  Picking 

Time  required  to  extract  and  QC 
anomaly  pick  locations  from  survey 
data 

0.5  hr/acre 

Parameter 

Estimation 

Time  required  to  extract  parameters  for 
all  anomalies. 

2  min/anomaly 
300  anom  (typ.) 
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9.0  IMPLEMENTATION  ISSUES 


The  primary  goal  of  this  project  was  to  demonstrate  a  UXO  classification  process  that  made  use 
of  dynamic  mode  data  collection  with  advanced  EMI  sensors.  The  data  collected  with  these 
systems  can  be  used  both  for  anomaly  detection  and  for  classification  on  a  significant  fraction  of 
the  detected  anomalies,  limiting  the  number  of  anomalies  requiring  cued  mode  investigation. 

At  a  line  spacing  of  40  cm,  one  pair  of  receiver  cubes  for  each  survey  line  passes  roughly  over 
the  same  path  as  a  pair  of  receiver  cubes  from  the  survey  line  on  either  side.  In  more  recent 
demonstrations,  this  spacing  has  been  increased  to  60  cm,  improving  the  production  rate.  For 
very  small  targets,  such  as  20mm  projectiles,  the  original  spacing  is  still  recommended. 

Another  ongoing  goal  has  been  to  transition  these  technologies  from  being  research  prototypes  to 
use  in  the  industrial  community  where  appropriate.  The  mechanics  of  collecting  classification- 
grade  advanced  EMI  cued  data  with  these  systems  have  been  shown  to  be  fairly  routine  in  the 
research  community.  As  part  of  the  ESTCP  Munitions  Response  Live  Site  Demonstrations, 
industrial  partners  have  been  exposed  to  the  MP  system  and  the  associated  data  collection  and 
processing  procedures.  The  success  of  this  effort  will  be  evaluated  on  an  ongoing  basis  through 
the  Live  Site  demonstrations.  In  the  past,  analysis  of  data  from  these  systems  has  been  somewhat 
of  a  specialty,  requiring  specific  software  and  knowledge  to  proficiently  conduct.  The  successful 
transition  of  the  TEMTADS  data  QC/analysis  process  to  the  Geosoft  Oasis  montaj  environment 
provides  a  clear  pathway  for  resolving  these  issues.  A  final  implementation  issue  is  that  a  clear 
path  to  making  the  MP  system  commercially  available  has  not  been  identified  yet.  Discussions 
with  various  groups  along  these  lines  are  ongoing. 
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